Skip to content

feat(writer): VortexWriter.finish() returns a WriteSummary (#664) - #707

Merged
dfa1 merged 1 commit into
mainfrom
write-summary
Nov 8, 2026
Merged

dfa1 merged 1 commit into
mainfrom
write-summary

Conversation

@dfa1

@dfa1 dfa1 commented Nov 8, 2026

Copy link
Copy Markdown
Owner

Closes #664. Last of three PRs (after #705, #706).

VortexWriter.finish() returns WriteSummary(fileSize, rowCount, columns); each column has compressedSize (its segments, as Rust's compressed_column_sizes) and stats (the footer's file statistics from #706, as ArrayStats from #705). Mirrors vortex-jni 0.90.0's VortexWriter.finish() / VortexWriteSummary.

  • close() = finish() unless it already ran (a second close() used to write a second footer).
  • writeChunk after finish() throws IllegalStateException; a failed finish() is not retried.
  • WriteSummaryInteropIntegrationTest: row count, null/NaN counts and bounds equal vortex-jni's summary for the same batches; fileSize and per-column sizes match the reopened file.

Naming: vortex-jni calls these VortexWriteSummary / VortexColumnStatistics; this keeps the shorter WriteSummary (like WriteOptions). Easy to rename if preferred.

🤖 Generated with Claude Code

close() is void, so a caller had to reopen the file to learn what it
wrote. finish() returns Rust's WriteSummary, as vortex-jni's
VortexWriter.finish() does: the file size, the row count and, per
top-level column, its compressed size and the footer's file statistics
(min, max, sum, null and NaN counts, folded per batch).

- The compressed size sums the column's segments (data, zone map,
  dictionary), Rust's compressed_column_sizes over its layout subtree.
- close() calls finish() unless it ran: a second close() no longer
  writes a second footer. writeChunk after finish() throws, as rows
  would land after the trailer; a failed finish() is not retried.

Cross-checked against vortex-jni's summary for the same batches (row
count, null/NaN counts, bounds), and the sizes against the reopened
file (WriteSummaryInteropIntegrationTest).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
@dfa1
dfa1 merged commit 15993cd into main Nov 8, 2026
6 checks passed
@dfa1
dfa1 deleted the write-summary branch November 8, 2026 12:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

writer: expose the statistics of file

1 participant